1. Field of the Invention
The present invention is related generally to a data processing system and in particular to a method and apparatus for compiling code. More particularly, the present invention is directed to a computer implemented method, apparatus, and computer usable program code for improving the running time of short running applications in a just-in-time environment by controlling interleaving of compilation with computation.
2. Description of the Related Art
Just-in-time (JIT) compilation, also referred to as dynamic translation, is a method for converting software code from one format into another during runtime. In one example, JIT compilation converts bytecode into native machine code.
Bytecode is a binary representation of program code that is an intermediate representation between source code and machine code. Bytecode is portable and can generally be compiled into machine code during runtime much faster than source code can be compiled into machine code. Bytecode may be converted into native machine language for execution of the native machine language and/or the bytecode may be executed directly on a virtual machine.
For software systems that compile applications using JIT compilation, compilation time is inherently part of the application's running time. Most of the compilation activity takes place during the initial stages of an application runtime. The time that it takes to compile the bytecode during the initial execution of the application is sometimes referred to as a startup time delay. Systems that compile software code using the JIT compiler work best with applications that run for a long time. These long running applications tend to minimize or amortize the startup time delay over a long run time period. However, the startup time delay in JIT compilation may result in a more noticeable or unacceptable delay that could hinder or negatively impact runtime performance in short running software applications and in the startup of big server applications.
Another problem with runtime compilation is that it may create a delay in the execution flow of an application. This makes graphical applications look jerky. It may also cause interactive applications to appear unresponsive. For all of these reasons, JIT compilers have traditionally refrained from using expensive or aggressive optimizations and relied primarily on cheaper or less aggressive optimizations that provide the best performance advantages for a given optimization.
The most common solution to address the compilation overhead problem is adaptive compilation. Most programs spend the vast majority of their time executing a few key routines. In this solution, the virtual machine (VM) identifies hot methods at runtime. A hot method, also referred to as a hot spot, refers to a part or portion of code, such as an algorithm, routine, or method, where a significant amount of time is being spent by the processor during execution of the code. In this solution, the virtual machine restricts JIT compilation activity to this small set of hot methods. In other words, the virtual machine compiles only the hot methods. The less important methods are interpreted rather than compiled by the JIT compiler at runtime.
As virtual machines evolved, they began to offer JIT compilers with multiple optimization levels. For instance, a virtual machine could provide a three level execution engine that includes an interpreter, a fast-non-optimizing compiler, and a slow optimizing compiler. Similarly, another solution provides three compilers with different capabilities: a baseline compiler that mimics a stack machine, a “quick” compiler that provides a very low level of optimization, and a full-fledged optimizing compiler. As another example, the IBM® J9 Virtual Machine (J9 VM) from International Business Machines Corporation offers an even higher granularity in terms of optimization levels. The hotter the method, the more optimizations are applied during compilation of the method.
These approaches can alleviate the overhead of compilation, but only if the application in question has clear hot-spots. For applications with flat execution profiles where most of the methods are more or less equally important, the benefit of dynamic compilation is seriously diminished. For example, in big server-side applications, like the Java™ 2 Enterprise Edition platform (J2EE) application servers, the sheer number of methods coupled with the absence of a clear hot-spot can significantly delay the startup of such applications. This is particularly important for servers that need to recover rapidly after a crash or power failure.
Short running applications can also be negatively affected by JIT compilation even in the presence of hot-spots because there may not be enough time to amortize the cost of compilation during runtime. It is not uncommon to spend a lot of time compiling only to have the application end soon thereafter. For example, if an application is very short, ninety percent (90%) of the runtime may be spent compiling the code and only ten percent (10%) of the runtime may be spent on computation tasks associated with running the actual application or performing the actual work the application is intended to perform. The end result is that a very large percentage of the application's time is spent compiling rather than performing useful work.